{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "SpaCy-informationextraction.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyMBWTg5N5Y3Uo6duje/bFUk",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/tomasonjo/blogs/blob/master/ie_pipeline/SpaCy_informationextraction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g2DBtIbZPRbL",
        "outputId": "71480621-b370-4a31-999d-ba2b6c93be78"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: crosslingual-coreference==0.2.3 in /usr/local/lib/python3.7/dist-packages (0.2.3)\n",
            "Requirement already satisfied: spacy-transformers==1.1.5 in /usr/local/lib/python3.7/dist-packages (1.1.5)\n",
            "Requirement already satisfied: wikipedia in /usr/local/lib/python3.7/dist-packages (1.4.0)\n",
            "Requirement already satisfied: neo4j in /usr/local/lib/python3.7/dist-packages (4.4.3)\n",
            "Requirement already satisfied: torchaudio<0.11 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (0.10.2)\n",
            "Requirement already satisfied: allennlp<3.0,>=2.8 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (2.9.3)\n",
            "Requirement already satisfied: allennlp-models<3.0,>=2.8 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (2.9.3)\n",
            "Requirement already satisfied: spacy<4.0,>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (3.2.4)\n",
            "Requirement already satisfied: torch<1.11.0,>=1.10.0 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (1.10.2)\n",
            "Requirement already satisfied: torchvision<0.12.0 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (0.11.3)\n",
            "Requirement already satisfied: click<8.1.0 in /usr/local/lib/python3.7/dist-packages (from crosslingual-coreference==0.2.3) (7.1.2)\n",
            "Requirement already satisfied: transformers<4.18.0,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers==1.1.5) (4.17.0)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers==1.1.5) (2.4.3)\n",
            "Requirement already satisfied: spacy-alignments<1.0.0,>=0.7.2 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers==1.1.5) (0.8.5)\n",
            "Requirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.3.4)\n",
            "Requirement already satisfied: requests>=2.18 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.23.0)\n",
            "Requirement already satisfied: fairscale==0.4.6 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.4.6)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.21.6)\n",
            "Requirement already satisfied: tensorboardX>=1.2 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.5)\n",
            "Requirement already satisfied: more-itertools in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (8.13.0)\n",
            "Requirement already satisfied: pytest in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.6.4)\n",
            "Requirement already satisfied: tqdm>=4.62 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.64.0)\n",
            "Requirement already satisfied: wandb<0.13.0,>=0.10.0 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.12.16)\n",
            "Requirement already satisfied: typer>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.4.1)\n",
            "Requirement already satisfied: termcolor==1.1.0 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.1.0)\n",
            "Requirement already satisfied: base58 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.1.1)\n",
            "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.1.96)\n",
            "Requirement already satisfied: h5py in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.1.0)\n",
            "Requirement already satisfied: filelock<3.7,>=3.3 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.6.0)\n",
            "Requirement already satisfied: lmdb in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.99)\n",
            "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.0.2)\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.7)\n",
            "Requirement already satisfied: jsonnet>=0.10.0 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.18.0)\n",
            "Requirement already satisfied: huggingface-hub>=0.0.16 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.5.1)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.4.1)\n",
            "Requirement already satisfied: cached-path<1.2.0,>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.1.2)\n",
            "Requirement already satisfied: word2number>=1.1 in /usr/local/lib/python3.7/dist-packages (from allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.1)\n",
            "Requirement already satisfied: ftfy in /usr/local/lib/python3.7/dist-packages (from allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (6.1.1)\n",
            "Requirement already satisfied: py-rouge==1.1 in /usr/local/lib/python3.7/dist-packages (from allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.1)\n",
            "Requirement already satisfied: conllu==4.4.1 in /usr/local/lib/python3.7/dist-packages (from allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.4.1)\n",
            "Requirement already satisfied: datasets in /usr/local/lib/python3.7/dist-packages (from allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.2.2)\n",
            "Requirement already satisfied: google-cloud-storage<3.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.3.0)\n",
            "Requirement already satisfied: boto3<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.23.5)\n",
            "Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /usr/local/lib/python3.7/dist-packages (from boto3<2.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.5.2)\n",
            "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from boto3<2.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.0.0)\n",
            "Requirement already satisfied: botocore<1.27.0,>=1.26.5 in /usr/local/lib/python3.7/dist-packages (from boto3<2.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.26.5)\n",
            "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/dist-packages (from botocore<1.27.0,>=1.26.5->boto3<2.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.8.2)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/local/lib/python3.7/dist-packages (from botocore<1.27.0,>=1.26.5->boto3<2.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.25.11)\n",
            "Requirement already satisfied: google-auth<3.0dev,>=1.25.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.35.0)\n",
            "Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.3.0)\n",
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.17.3)\n",
            "Requirement already satisfied: google-resumable-media>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.3.3)\n",
            "Requirement already satisfied: google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.31.5)\n",
            "Requirement already satisfied: six>=1.13.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.15.0)\n",
            "Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (21.3)\n",
            "Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2022.1)\n",
            "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (57.4.0)\n",
            "Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.56.1)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3.0dev,>=1.25.0->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.8)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3.0dev,>=1.25.0->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.2.4)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3.0dev,>=1.25.0->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.2.8)\n",
            "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib/python3.7/dist-packages (from google-resumable-media>=2.3.2->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.3.0)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from huggingface-hub>=0.0.16->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.11.3)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub>=0.0.16->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.10.0.2)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from huggingface-hub>=0.0.16->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (6.0)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.7/dist-packages (from nltk->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2022.4.24)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from nltk->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.1.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=14.3->google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.0.9)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3.0dev,>=1.25.0->google-cloud-storage<3.0,>=1.0->cached-path<1.2.0,>=1.0.2->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.4.8)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.18->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.18->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2021.10.8)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.18->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.0.4)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (1.8.2)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (0.4.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (2.11.3)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (0.9.1)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (0.6.1)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (2.0.7)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (3.0.6)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (1.0.7)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (1.0.2)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (3.0.9)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (2.0.6)\n",
            "Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (8.0.16)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (3.3.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (3.8.0)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (5.2.1)\n",
            "Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision<0.12.0->crosslingual-coreference==0.2.3) (7.1.2)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,>=0.11.1 in /usr/local/lib/python3.7/dist-packages (from transformers<4.18.0,>=3.4.0->spacy-transformers==1.1.5) (0.12.1)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers<4.18.0,>=3.4.0->spacy-transformers==1.1.5) (0.0.53)\n",
            "Requirement already satisfied: setproctitle in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.2.3)\n",
            "Requirement already satisfied: docker-pycreds>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.4.0)\n",
            "Requirement already satisfied: shortuuid>=0.5.0 in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.0.9)\n",
            "Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (5.4.8)\n",
            "Requirement already satisfied: promise<3,>=2.0 in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.3)\n",
            "Requirement already satisfied: sentry-sdk>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.5.12)\n",
            "Requirement already satisfied: GitPython>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.1.27)\n",
            "Requirement already satisfied: pathtools in /usr/local/lib/python3.7/dist-packages (from wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.1.2)\n",
            "Requirement already satisfied: gitdb<5,>=4.0.1 in /usr/local/lib/python3.7/dist-packages (from GitPython>=1.0.0->wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.0.9)\n",
            "Requirement already satisfied: smmap<6,>=3.0.1 in /usr/local/lib/python3.7/dist-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb<0.13.0,>=0.10.0->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (5.0.0)\n",
            "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.7/dist-packages (from wikipedia) (4.6.3)\n",
            "Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (2022.5.0)\n",
            "Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (6.0.1)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.8.1)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.0.0)\n",
            "Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.18.0)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.70.12.2)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.3.5)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.7.2)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (6.0.2)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (4.0.2)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.2.0)\n",
            "Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.13.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (21.4.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.3.0)\n",
            "Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (2.0.12)\n",
            "Requirement already satisfied: wcwidth>=0.2.5 in /usr/local/lib/python3.7/dist-packages (from ftfy->allennlp-models<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.2.5)\n",
            "Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.5.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy<4.0,>=3.1.0->crosslingual-coreference==0.2.3) (2.0.1)\n",
            "Requirement already satisfied: py>=1.5.0 in /usr/local/lib/python3.7/dist-packages (from pytest->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.11.0)\n",
            "Requirement already satisfied: pluggy<0.8,>=0.5 in /usr/local/lib/python3.7/dist-packages (from pytest->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (0.7.1)\n",
            "Requirement already satisfied: atomicwrites>=1.0 in /usr/local/lib/python3.7/dist-packages (from pytest->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (1.4.0)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->allennlp<3.0,>=2.8->crosslingual-coreference==0.2.3) (3.1.0)\n",
            "Requirement already satisfied: google-cloud-storage in /usr/local/lib/python3.7/dist-packages (2.3.0)\n",
            "Requirement already satisfied: google-resumable-media>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (2.3.3)\n",
            "Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (2.3.0)\n",
            "Requirement already satisfied: google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (1.31.5)\n",
            "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (2.23.0)\n",
            "Requirement already satisfied: google-auth<3.0dev,>=1.25.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (1.35.0)\n",
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage) (3.17.3)\n",
            "Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage) (1.56.1)\n",
            "Requirement already satisfied: six>=1.13.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage) (1.15.0)\n",
            "Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage) (2022.1)\n",
            "Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage) (21.3)\n",
            "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage) (57.4.0)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3.0dev,>=1.25.0->google-cloud-storage) (4.2.4)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3.0dev,>=1.25.0->google-cloud-storage) (4.8)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3.0dev,>=1.25.0->google-cloud-storage) (0.2.8)\n",
            "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib/python3.7/dist-packages (from google-resumable-media>=2.3.2->google-cloud-storage) (1.3.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=14.3->google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage) (3.0.9)\n",
            "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3.0dev,>=1.25.0->google-cloud-storage) (0.4.8)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage) (2021.10.8)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage) (1.25.11)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage) (3.0.4)\n",
            "Collecting transformers==4.18.0\n",
            "  Using cached transformers-4.18.0-py3-none-any.whl (4.0 MB)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (3.6.0)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (2022.4.24)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (2.23.0)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (0.12.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (21.3)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (6.0)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (4.64.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (1.21.6)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (0.5.1)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (4.11.3)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers==4.18.0) (0.0.53)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers==4.18.0) (3.10.0.2)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers==4.18.0) (3.0.9)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers==4.18.0) (3.8.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==4.18.0) (2021.10.8)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==4.18.0) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==4.18.0) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers==4.18.0) (1.25.11)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers==4.18.0) (1.15.0)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers==4.18.0) (7.1.2)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers==4.18.0) (1.1.0)\n",
            "Installing collected packages: transformers\n",
            "  Attempting uninstall: transformers\n",
            "    Found existing installation: transformers 4.17.0\n",
            "    Uninstalling transformers-4.17.0:\n",
            "      Successfully uninstalled transformers-4.17.0\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "spacy-transformers 1.1.5 requires transformers<4.18.0,>=3.4.0, but you have transformers 4.18.0 which is incompatible.\u001b[0m\n",
            "Successfully installed transformers-4.18.0\n",
            "Collecting en-core-web-sm==3.2.0\n",
            "  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.2.0/en_core_web_sm-3.2.0-py3-none-any.whl (13.9 MB)\n",
            "\u001b[K     |████████████████████████████████| 13.9 MB 10.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: spacy<3.3.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from en-core-web-sm==3.2.0) (3.2.4)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (0.4.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (21.3)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.3.0)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (1.0.2)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (0.9.1)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (57.4.0)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (1.0.7)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.23.0)\n",
            "Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (8.0.16)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (1.21.6)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.0.7)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.0.9)\n",
            "Requirement already satisfied: click<8.1.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (7.1.2)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.4.3)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (0.6.1)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (0.4.1)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (1.8.2)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (4.64.0)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.10.0.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.11.3)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.8.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.0.9)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (5.2.1)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (1.25.11)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (3.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2021.10.8)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.10)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy<3.3.0,>=3.2.0->en-core-web-sm==3.2.0) (2.0.1)\n",
            "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
            "You can now load the package via spacy.load('en_core_web_sm')\n"
          ]
        }
      ],
      "source": [
        "!pip install crosslingual-coreference==0.2.3 spacy-transformers==1.1.5 wikipedia neo4j\n",
        "!pip install --upgrade google-cloud-storage\n",
        "!pip install transformers==4.18.0\n",
        "!python -m spacy download en_core_web_sm\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Restart runtime"
      ],
      "metadata": {
        "id": "hJfrLdtEIR9s"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Since I first dabbled with natural language processing, I have had a special place in my heart for information extraction (IE) pipelines. Information extraction (IE) pipelines extract structured data from unstructured data like text. The internet provides an abundance of information in the form of various articles and other content formats. However, while you might read the news or subscribe to multiple podcasts, it is virtually impossible to keep track of all the new information released daily. Even if you could manually read all the latest reports and articles, it would be incredibly tedious and labor-intensive to structure the data so that you can easily query and aggregate it with your preferred tools. I definitely wouldn't want to be doing that as my job. Luckily, we can resort to using the latest state-of-the-art NLP techniques to do the information extraction for us automatically.\n",
        "\n",
        "# Developing IE pipeline in SpaCy\n",
        "There has been much development around [SpaCy](https://spacy.io/) in the last couple of weeks, so I decided to try out the new plugins and use them to construct an information extraction pipeline.\n",
        "\n",
        "## Coreference resolution\n",
        "First off, we are going to be using the new [Crosslingual Coreference](https://spacy.io/universe/project/crosslingualcoreference) model contributed by [David Berenstein](https://www.linkedin.com/in/david-berenstein-1bab11105/) to the SpaCy Universe. SpaCy Universe is a collection of open-source plugins or addons for SpaCy. The cool thing about the SpaCy universe project is that it's straightforward to add the models to our pipeline.\n",
        "\n",
        "## Relation extraction\n",
        "You might wonder why we skipped the named entity recognition and linking step. Well, the reason is that we will be using the [Rebel project](https://github.com/Babelscape/rebel) that recognizes both the entities and relations from the text. If I understand correctly, the Rebel project was developed by [Pere-Lluís Huguet Cabot](https://www.linkedin.com/in/perelluis/) as a part of his PhD study. Again, a massive shoutout to Pere for creating such an incredible library with state-of-the-art results for relation extraction. The Rebel model is available on Hugginface as well as in the form of SpaCy component.\n",
        "However, the model doesn't do any entity linking, so we will implement our version of entity linking. We will simply search for entities on WikiData by calling the search entities WikiData API.\n",
        "\n",
        "I've been looking at how to improve entity linking and stumbled upon the [ExtEnd project](https://github.com/SapienzaNLP/extend). ExtEnd project is a novel approach to entity disambiguation and is available as a demo on Huggingface as well as a SpaCy component. I've played around with it a bit and managed to get it working using WikiData API for candidates instead of the original AIDA candidates. However, when I wanted to have all three projects (Coref, Rebel, ExtEnd) in the same pipeline, there were some dependency issues as they use a different version of PyTorch, so I gave up for now. However, the ExtEnd code I developed is available on GitHub, and if someone wants to help me get it working, I am more than happy to accept Pull requests."
      ],
      "metadata": {
        "id": "0truD94Fb_Du"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import spacy\n",
        "import crosslingual_coreference"
      ],
      "metadata": {
        "id": "BrcXpPzTSszu",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4af118e9-57bc-4313-f930-9c2348d2f219"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package omw-1.4 to /root/nltk_data...\n",
            "[nltk_data]   Package omw-1.4 is already up-to-date!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Ok, so for now, we won't be using the ExtEnd project but will use a simplified version of entity linking by simply taking the first candidate fetched from the WikiData API. The only thing we need to do is incorporate our simplified entity linking solution in the Rebel pipeline. Since the Rebel component is not available directly as a SpaCy Universe project, we must copy the component definition from their repository manually. I've taken the liberty to implement my version of the set_annotations function in the Rebel SpaCy component, while the rest of the code is the same as the original."
      ],
      "metadata": {
        "id": "PEWS7OiCcUxo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Add rebel component https://github.com/Babelscape/rebel/blob/main/spacy_component.py\n",
        "import requests\n",
        "import re\n",
        "import hashlib\n",
        "from spacy import Language\n",
        "from typing import List\n",
        "\n",
        "from spacy.tokens import Doc, Span\n",
        "\n",
        "from transformers import pipeline\n",
        "\n",
        "def call_wiki_api(item):\n",
        "  try:\n",
        "    url = f\"https://www.wikidata.org/w/api.php?action=wbsearchentities&search={item}&language=en&format=json\"\n",
        "    data = requests.get(url).json()\n",
        "    # Return the first id (Could upgrade this in the future)\n",
        "    return data['search'][0]['id']\n",
        "  except:\n",
        "    return 'id-less'\n",
        "\n",
        "def extract_triplets(text):\n",
        "    \"\"\"\n",
        "    Function to parse the generated text and extract the triplets\n",
        "    \"\"\"\n",
        "    triplets = []\n",
        "    relation, subject, relation, object_ = '', '', '', ''\n",
        "    text = text.strip()\n",
        "    current = 'x'\n",
        "    for token in text.replace(\"<s>\", \"\").replace(\"<pad>\", \"\").replace(\"</s>\", \"\").split():\n",
        "        if token == \"<triplet>\":\n",
        "            current = 't'\n",
        "            if relation != '':\n",
        "                triplets.append({'head': subject.strip(), 'type': relation.strip(),'tail': object_.strip()})\n",
        "                relation = ''\n",
        "            subject = ''\n",
        "        elif token == \"<subj>\":\n",
        "            current = 's'\n",
        "            if relation != '':\n",
        "                triplets.append({'head': subject.strip(), 'type': relation.strip(),'tail': object_.strip()})\n",
        "            object_ = ''\n",
        "        elif token == \"<obj>\":\n",
        "            current = 'o'\n",
        "            relation = ''\n",
        "        else:\n",
        "            if current == 't':\n",
        "                subject += ' ' + token\n",
        "            elif current == 's':\n",
        "                object_ += ' ' + token\n",
        "            elif current == 'o':\n",
        "                relation += ' ' + token\n",
        "    if subject != '' and relation != '' and object_ != '':\n",
        "        triplets.append({'head': subject.strip(), 'type': relation.strip(),'tail': object_.strip()})\n",
        "\n",
        "    return triplets\n",
        "\n",
        "\n",
        "@Language.factory(\n",
        "    \"rebel\",\n",
        "    requires=[\"doc.sents\"],\n",
        "    assigns=[\"doc._.rel\"],\n",
        "    default_config={\n",
        "        \"model_name\": \"Babelscape/rebel-large\",\n",
        "        \"device\": 0,\n",
        "    },\n",
        ")\n",
        "class RebelComponent:\n",
        "    def __init__(\n",
        "        self,\n",
        "        nlp,\n",
        "        name,\n",
        "        model_name: str,\n",
        "        device: int,\n",
        "    ):\n",
        "        assert model_name is not None, \"\"\n",
        "        self.triplet_extractor = pipeline(\"text2text-generation\", model=model_name, tokenizer=model_name, device=device)\n",
        "        self.entity_mapping = {}\n",
        "        # Register custom extension on the Doc\n",
        "        if not Doc.has_extension(\"rel\"):\n",
        "          Doc.set_extension(\"rel\", default={})\n",
        "\n",
        "    def get_wiki_id(self, item: str):\n",
        "        mapping = self.entity_mapping.get(item)\n",
        "        if mapping:\n",
        "          return mapping\n",
        "        else:\n",
        "          res = call_wiki_api(item)\n",
        "          self.entity_mapping[item] = res\n",
        "          return res\n",
        "\n",
        "    \n",
        "    def _generate_triplets(self, sent: Span) -> List[dict]:\n",
        "          output_ids = self.triplet_extractor(sent.text, return_tensors=True, return_text=False)[0][\"generated_token_ids\"][\"output_ids\"]\n",
        "          extracted_text = self.triplet_extractor.tokenizer.batch_decode(output_ids[0])\n",
        "          extracted_triplets = extract_triplets(extracted_text[0])\n",
        "          return extracted_triplets\n",
        "\n",
        "    def set_annotations(self, doc: Doc, triplets: List[dict]):\n",
        "        for triplet in triplets:\n",
        "\n",
        "            # Remove self-loops (relationships that start and end at the entity)\n",
        "            if triplet['head'] == triplet['tail']:\n",
        "                continue\n",
        "\n",
        "            # Use regex to search for entities\n",
        "            head_span = re.search(triplet[\"head\"], doc.text)\n",
        "            tail_span = re.search(triplet[\"tail\"], doc.text)\n",
        "\n",
        "            # Skip the relation if both head and tail entities are not present in the text\n",
        "            # Sometimes the Rebel model hallucinates some entities\n",
        "            if not head_span or not tail_span:\n",
        "              continue\n",
        "\n",
        "            index = hashlib.sha1(\"\".join([triplet['head'], triplet['tail'], triplet['type']]).encode('utf-8')).hexdigest()\n",
        "            if index not in doc._.rel:\n",
        "                # Get wiki ids and store results\n",
        "                doc._.rel[index] = {\"relation\": triplet[\"type\"], \"head_span\": {'text': triplet['head'], 'id': self.get_wiki_id(triplet['head'])}, \"tail_span\": {'text': triplet['tail'], 'id': self.get_wiki_id(triplet['tail'])}}\n",
        "\n",
        "    def __call__(self, doc: Doc) -> Doc:\n",
        "        for sent in doc.sents:\n",
        "            sentence_triplets = self._generate_triplets(sent)\n",
        "            self.set_annotations(doc, sentence_triplets)\n",
        "        return doc"
      ],
      "metadata": {
        "id": "cGsgtd8lxhVo"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The set_annotations function handles how we store the results back to the SpaCy's Doc object. First, we ignore all the self-loops. Self-loops are relationships that start and end at the same entity. Next, we search for both the head and tail entities of the relation in the text using regex. I've noticed that the Rebel model sometimes hallucinates some entities which are not in the original text. For that reason, I added a step that verifies that both entities are actually in the text before appending them to the results.\n",
        "Lastly, we use the WikiData API to map extracted entities to WikiData ids. As mentioned, this is a simplified version of entity disambiguation and linking, and you can take a more novel approach like the ExtEnd model, for example.\n",
        "\n",
        "Now that the Rebel SpaCy component is defined, we can create a two SpaCy pipelines to handle coreference resolution as well as relation extraction and entity linking."
      ],
      "metadata": {
        "id": "0U--nmKZcY7t"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "DEVICE = -1 # Number of the GPU, -1 if want to use CPU\n",
        "\n",
        "# Add coreference resolution model\n",
        "coref = spacy.load('en_core_web_sm', disable=['ner', 'tagger', 'parser', 'attribute_ruler', 'lemmatizer'])\n",
        "coref.add_pipe(\n",
        "    \"xx_coref\", config={\"chunk_size\": 2500, \"chunk_overlap\": 2, \"device\": DEVICE})\n",
        "\n",
        "# Define rel extraction model\n",
        "\n",
        "rel_ext = spacy.load('en_core_web_sm', disable=['ner', 'lemmatizer', 'attribute_rules', 'tagger'])\n",
        "rel_ext.add_pipe(\"rebel\", config={\n",
        "    'device':DEVICE, # Number of the GPU, -1 if want to use CPU\n",
        "    'model_name':'Babelscape/rebel-large'} # Model used, will default to 'Babelscape/rebel-large' if not given\n",
        "    )\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SSTDfj3nSSca",
        "outputId": "0dbdbb3c-7a6c-46a3-8b9e-dc89f8af7486"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the model checkpoint at nreimers/mMiniLMv2-L12-H384-distilled-from-XLMR-Large were not used when initializing XLMRobertaModel: ['lm_head.dense.bias', 'lm_head.layer_norm.bias', 'lm_head.dense.weight', 'lm_head.bias', 'lm_head.layer_norm.weight']\n",
            "- This IS expected if you are initializing XLMRobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing XLMRobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of XLMRobertaModel were not initialized from the model checkpoint at nreimers/mMiniLMv2-L12-H384-distilled-from-XLMR-Large and are newly initialized: ['roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<__main__.RebelComponent at 0x7f5a7f0eef10>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "input_text = \"Christian Drosten works in Germany. He likes to work for Google.\"\n",
        "\n",
        "coref_text = coref(input_text)._.resolved_text\n",
        "\n",
        "doc = rel_ext(coref_text)\n",
        "\n",
        "for value, rel_dict in doc._.rel.items():\n",
        "    print(f\"{value}: {rel_dict}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-uEgInLc-X3n",
        "outputId": "4e788c98-1173-4b36-e085-6687845638a9"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "385d9ac30c35635c7e36a1636ded4a091f830cf3: {'relation': 'country of citizenship', 'head_span': {'text': 'Christian Drosten', 'id': 'Q1079331'}, 'tail_span': {'text': 'Germany', 'id': 'Q183'}}\n",
            "471a35571b66cc8e1e3415e27f7086505310efc6: {'relation': 'employer', 'head_span': {'text': 'Christian Drosten', 'id': 'Q1079331'}, 'tail_span': {'text': 'Google', 'id': 'Q95'}}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The Rebel model extracted two relations from the text. For example, it recognized that Christian Drosten, with the WikiData id Q1079331, is employed by Google, which has an id Q95.\n",
        "# Storing the information extraction pipeline results\n",
        "Whenever I hear about relation information between entities, I think of a graph. A graph database is developed to store relations between entities, so what better fit to store the information extraction pipeline results.\n",
        "As you might know, I am biased towards Neo4j, but you can use whatever tool you like. Here, I will demonstrate how to store the results of my implementation of the information extraction pipeline into Neo4j. We will process a couple of Wikipedia summaries of famous women scientists and store the results as a graph.\n",
        "If you want to follow with the code examples, I would suggest you to create a [Blank Project in Neo4j Sandbox](https://sandbox.neo4j.com/?usecase=blank-sandbox) environment. After you have created the Neo4j Sandbox instance, you can copy the credentials to the code."
      ],
      "metadata": {
        "id": "ShXY3PF4cmLk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import wikipedia\n",
        "from neo4j import GraphDatabase\n",
        "\n",
        "# Define Neo4j connection\n",
        "host = 'bolt://3.236.134.179:7687'\n",
        "user = 'neo4j'\n",
        "password = 'writer-calibers-steels'\n",
        "driver = GraphDatabase.driver(host,auth=(user, password))\n",
        "\n",
        "import_query = \"\"\"\n",
        "UNWIND $data AS row\n",
        "MERGE (h:Entity {id: CASE WHEN NOT row.head_span.id = 'id-less' THEN row.head_span.id ELSE row.head_span.text END})\n",
        "ON CREATE SET h.text = row.head_span.text\n",
        "MERGE (t:Entity {id: CASE WHEN NOT row.tail_span.id = 'id-less' THEN row.tail_span.id ELSE row.tail_span.text END})\n",
        "ON CREATE SET t.text = row.tail_span.text\n",
        "WITH row, h, t\n",
        "CALL apoc.merge.relationship(h, toUpper(replace(row.relation,' ', '_')),\n",
        "  {},\n",
        "  {},\n",
        "  t,\n",
        "  {}\n",
        ")\n",
        "YIELD rel\n",
        "RETURN distinct 'done' AS result;\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "def run_query(query, params={}):\n",
        "    with driver.session() as session:\n",
        "        result = session.run(query, params)\n",
        "        return pd.DataFrame([r.values() for r in result], columns=result.keys())\n",
        "\n",
        "def store_wikipedia_summary(page):\n",
        "  try:\n",
        "    input_text = wikipedia.page(page).summary\n",
        "    coref_text = coref(input_text)._.resolved_text\n",
        "    doc = rel_ext(coref_text)\n",
        "    params = [rel_dict for value, rel_dict in doc._.rel.items()]\n",
        "    run_query(import_query, {'data': params})\n",
        "  except Exception as e:\n",
        "    print(f\"Couldn't parse text for {page} due to {e}\")\n"
      ],
      "metadata": {
        "id": "4lYZUb-CefzH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We've used the wikipedia python library to help us fetch the summaries from Wikipedia. Next, we need to define the Cypher statement used to import the information extraction results. I won't go into details of Cypher syntax, but basically, we first merge the head and tail entities by their WikiData id and then use a procedure from the APOC library to merge the relationship. I recommend going through courses in the Neo4j Graph Academy if you are looking for resources to learn more about Cypher syntax.\n",
        "\n",
        "Now that we have everything ready, we can go ahead and parse a couple of Wikipedia summaries."
      ],
      "metadata": {
        "id": "C-JNNxbTcrcz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ladies = [\"Jennifer Doudna\", \"Rachel Carson\", \"Sara Seager OC\", \"Gertrude Elion\", \"Rita Levi-Montalcini\"]\n",
        "\n",
        "for l in ladies:\n",
        "  print(f\"Parsing {l}\")\n",
        "  store_wikipedia_summary(l)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nVD9mMgfXJ7Z",
        "outputId": "b91fc7ee-ff3a-4abf-d777-5dedad13e57d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Parsing Jennifer Doudna\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/allennlp/modules/token_embedders/pretrained_transformer_embedder.py:385: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').\n",
            "  num_effective_segments = (seq_lengths + self._max_length - 1) // self._max_length\n",
            "/usr/local/lib/python3.7/dist-packages/transformers/pipelines/base.py:998: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset\n",
            "  UserWarning,\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Parsing Rachel Carson\n",
            "Parsing Sara Seager OC\n",
            "Parsing Gertrude Elion\n",
            "Parsing Rita Levi-Montalcini\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Enriching the graph\n",
        "Since we have mapped our entities to the WikiData ids, we can further use the WikiData API to enrich our graph. I will show you how to extract INSTANCE_OF relations from WikiData and store them to Neo4j with the help of the APOC library, which allows us to call web APIs and store results in the database.\n",
        "To be able to call WikiData API, you need to have a basic understanding of SPARQL syntax, but that is beyond the scope of this blog post. However, I've written a [post some time about that shows more SPARQL queries used to enrich a Neo4j graph and delves into SPARQL syntax](https://towardsdatascience.com/lord-of-the-wiki-ring-importing-wikidata-into-neo4j-and-analyzing-family-trees-da27f64d675e).\n",
        "By executing the following query, we add the Class nodes to the graph and link them with the appropriate entities."
      ],
      "metadata": {
        "id": "uwkDgt1Ecwxd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "run_query(\"\"\"\n",
        "CALL apoc.periodic.iterate(\"\n",
        "  MATCH (e:Entity)\n",
        "  WHERE e.id STARTS WITH 'Q'\n",
        "  RETURN e\n",
        "\",\"\n",
        "  // Prepare a SparQL query\n",
        "  WITH 'SELECT * WHERE{ ?item rdfs:label ?name . filter (?item = wd:' + e.id + ') filter (lang(?name) = \\\\\\\"en\\\\\\\") ' +\n",
        "     'OPTIONAL {?item wdt:P31 [rdfs:label ?label] .filter(lang(?label)=\\\\\\\"en\\\\\\\")}}' AS sparql, e\n",
        "  // make a request to Wikidata\n",
        "  CALL apoc.load.jsonParams(\n",
        "    'https://query.wikidata.org/sparql?query=' + \n",
        "      sparql,\n",
        "      { Accept: 'application/sparql-results+json'}, null)\n",
        "  YIELD value\n",
        "  UNWIND value['results']['bindings'] as row\n",
        "  SET e.wikipedia_name = row.name.value\n",
        "  WITH e, row.label.value AS label\n",
        "  MERGE (c:Class {id:label})\n",
        "  MERGE (e)-[:INSTANCE_OF]->(c)\n",
        "  RETURN distinct 'done'\", {batchSize:1, retry:1})\n",
        "\"\"\")"
      ],
      "metadata": {
        "id": "qk41QfxBmDRq",
        "outputId": "def62e81-2e10-4038-d71e-f0f46ccd1833",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   batches  total  timeTaken  committedOperations  failedOperations  \\\n",
              "0       54     54         10                   49                 5   \n",
              "\n",
              "   failedBatches  retries                                      errorMessages  \\\n",
              "0              5        0  {'Cannot merge the following node because of n...   \n",
              "\n",
              "                                               batch  \\\n",
              "0  {'total': 54, 'committed': 49, 'failed': 5, 'e...   \n",
              "\n",
              "                                          operations  wasTerminated  \\\n",
              "0  {'total': 54, 'committed': 49, 'failed': 5, 'e...          False   \n",
              "\n",
              "  failedParams                                   updateStatistics  \n",
              "0           {}  {'nodesDeleted': 0, 'labelsAdded': 58, 'relati...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9f91f89c-981e-48ee-8175-bc90e9dda39f\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>batches</th>\n",
              "      <th>total</th>\n",
              "      <th>timeTaken</th>\n",
              "      <th>committedOperations</th>\n",
              "      <th>failedOperations</th>\n",
              "      <th>failedBatches</th>\n",
              "      <th>retries</th>\n",
              "      <th>errorMessages</th>\n",
              "      <th>batch</th>\n",
              "      <th>operations</th>\n",
              "      <th>wasTerminated</th>\n",
              "      <th>failedParams</th>\n",
              "      <th>updateStatistics</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>54</td>\n",
              "      <td>54</td>\n",
              "      <td>10</td>\n",
              "      <td>49</td>\n",
              "      <td>5</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>{'Cannot merge the following node because of n...</td>\n",
              "      <td>{'total': 54, 'committed': 49, 'failed': 5, 'e...</td>\n",
              "      <td>{'total': 54, 'committed': 49, 'failed': 5, 'e...</td>\n",
              "      <td>False</td>\n",
              "      <td>{}</td>\n",
              "      <td>{'nodesDeleted': 0, 'labelsAdded': 58, 'relati...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9f91f89c-981e-48ee-8175-bc90e9dda39f')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-9f91f89c-981e-48ee-8175-bc90e9dda39f button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-9f91f89c-981e-48ee-8175-bc90e9dda39f');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Conclusion\n",
        "I really like what SpaCy is doing lately and all the open-source projects around it. I noticed that various open-source projects are primarily standalone, and it can be tricky to combine multiple models into a single SpaCy pipeline. For example, you can see that we had to have two pipelines in this project, one for coreference resolution and one for relation extraction and entity linking.\n",
        "\n",
        "As for the results of the IE pipeline, I am delighted with how well it turned out. As you can observe in the Rebel repository, their solution is state-of-the-art on many NLP datasets, so it is not a big surprise that the results are so good. The only weak link in my implementation is the Entity Linking step. As I said, it would probably greatly benefit from adding something like the ExtEnd library for more accurate entity disambiguation and linking. Perhaps that's something I'll do next time.\n",
        "\n",
        "Try out the IE implementation, and please let me know what you think or if you have some ideas for improvements. There are a ton of opportunities to make this pipeline better!"
      ],
      "metadata": {
        "id": "wFiNO_f9c1d7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "XvdDiBgkodMU"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}